A Universal Criteria Catalog for Evaluation of Heterogeneous Agent Development Artifacts

The research discipline of multi-agent systems is characterized by a high degree of heterogeneity. This heterogeneity leads to a vast amount of options (e.g. different architectures and languages) how to employ agent technology but is also one major source of difficulties for its adoption. People interested in using multi-agent systems depend on solid survey articles, which clarify and evaluate these different options and explain in which situations which choices should be made. A survey should also propose viable classification means for helping readers to understand which development artifacts broadly exhibit similar properties. To date, in most cases multi-agent system surveys do without classifications and only address one specific type of artifact such as agent languages or tools. Often, only the characteristics of the representatives are described without evaluating them. In this work a universal criteria catalog will be presented that has been defined abstractly enough for being usable for a wide variety of agent development artifacts. It will be shown how this abstract catalog can be further refined with respect to the chosen area of investigation. In addition to the catalog its general usage as part of a survey will be explained and a blueprint for survey conduction will be presented. To demonstrate its usefulness cutouts of extensive evaluations, performed in the areas of agent architectures, languages, methodologies, tools and platforms, will be presented.